Several approaches have been made to perform recognition of human motion, with emphasis on real-time computation. In addition, several survey papers have reviewed vision-based motion recognition, human motion capture, and human-motion analysis. The most frequently used methodology for recognition of body motion and dynamic gestures was based on the analysis of temporal trajectories of the motion parameters, hidden Markov models and state-space models, and static-activity templates.
Other conventional techniques have attempted to represent motion to describe an action sequence by collecting of optical flow over the image or region of interest throughout the sequence, but this is computationally expensive and often was not robust. Another conventional technique combined several successive layers of image frames of a moving person in a single template. This single template represented temporal history of a motion, allowing a match of actual imagery to a memorized template to produce recognition of a motion gesture.
Further, all of the above conventional techniques have been conducted in controlled laboratory environments, with fixed lighting and constant distance to the subject. Obviously, actual field conditions will present external stimuli, and resulting difficulties in recognition. Thus, it is apparent that different approaches are required for real-life applications outside of a lab, such as the flight deck of an aircraft carrier.
Flight deck operations are a “dance of chaos” with steam, constant motion, and crowding. These operations are conducted during day or night, and rain or snow, when visibility is extremely poor in the unforgiving maritime environment. Moreover, fleet operations are continually subject to reduced manning and resistance to change. It is desired that, in these kinds of conditions, Unmanned Combat Air Vehicles (UCAV) shall be launched from the flight decks of aircraft carriers.
In order to launch a UCAV from a flight deck, the UCAV must be controlled during taxiing somehow before takeoff, and after landing. Simply hooking a tow tractor to the aircraft has been considered, but was deemed too slow, especially since the aircraft are required to recover every 45 seconds and need to taxi out of the landing area for the next aircraft. Alternatively, providing the aircraft director/controller with a joystick to control the aircraft would tax his/her workload with an additional process, and would negatively impact training and operations.
Further, if the UCAV is to be controlled on deck using a joystick, the UCAV would necessarily be controlled vi radio (RF) link. However, a RF link from a control device to the UAV is undesirable because of the EMI (electromagnetic interference) intensive environment on the flight deck, and the EMCON constraints. Another alternative is a tethered connection, using a control device physically tethered to the UCAV. However, such a tethered connection may be potentially unsafe for the personnel on the deck during high tempo operations.
Just like manned aircraft, the UCAVs taxi before launch, after recoveries or during re-spotting. In the case of manned aircraft, flight deck controllers signal directions to pilots for taxiing the aircraft around the deck or airfield. It would be most desirable if these signals were used to develop an automatic taxiing system for unmanned aircraft as well. If such a system were developed, it would enable a seamless transition of UCAVs into naval aviation.
It is an object of the present invention to overcome the difficulties discussed above, and provide a system and software program for use in such a system to allow a user to remotely control a robotic device using merely gestures or motion signals.
Further, it is an object of the present invention to overcome the difficulties discussed above using a machine vision based approach, which would least impact operations and training and therefore held the most promise from the operational point-of-view.
It is another object of the present invention to provide a system as described above, using sensor(s) mounted on the robotic device, in conjunction with image recognition software residing on an onboard computer, to provide inputs to the robotic devices' control system.